Selective Functional Transfer: Inductive Bias from Related Tasks

The selective transfer of task knowledge within the context of artificial neural networks is studied in the MTL learning framework, a modified version of the multiple task learning (MTL) method of functional transfer. MTL is a knowledge based inductive learning system that uses prior task knowledge to adjust its inductive bias. MTL employs a separate learning rate for each task output node. The learning rate for each secondary task varies as a function of a measure of relatedness between that task and the primary task. A definition of task relatedness is given. Eight task relatedness measures are presented and are compared empirically. Experiments demonstrate that from impoverished training sets MTL develops predictive models which have superior generalization ability compared with models produced by single task learning or multiple task learning.